The present invention relates to an image processing apparatus and an image processing method, for processing digitized image data. More specifically, the present invention is directed to an image producing apparatus and an image producing method, capable of realizing various sorts of image processing functions by preparing a plurality of element/functional image processing modules and by combining these image processing modules with each other.
Techniques capable of providing various sorts of image processing functions by preparing a plurality of element/functional image processing modules and by combing these element/functional image processing modules and by combining thereof with each other are utilized in various systems capable of handling digital images, for example, document editors, drawing tools, image transfer apparatus, printers, and the like. Then, such image processing functions may be provided as image processing libraries in some cases. In the case of these image processing libraries, functions thereof may be utilized by linking the image processing libraries to application programs by which the image processing functions are wanted to be utilized.
In DTP (desk top publishing) systems capable of inputting images, and print systems capable of outputting images, various sorts of image processing operations are carried out with respect to images which should be processed. In these image processing operations, for instance, enlarging operations, reducing operations, rotating operations, affine transformations, color conversions, filtering processes, synthesizing operations, and the like are carried out. When these image processing operations are carried out, these image processing operations may be carried out by employing dedicated hardware in such a case that attributes, image processed contents, sequential orders, parameters of input images are fixed. However, for instance, in such a case that various types (different color spaces, and different bit numbers per bit) are entered, and/or sequential orders of image processed contents and parameters thereof are changed, these image processing operations must be carried out by properly employing image-processing arrangements having more flexible characteristics.
As means capable of satisfying such a requirement, several conventional techniques are proposed, by which desirable image processing operations can be carried out in flexible manners while programmable modules are connected to each other in pipeline modes and/or DAG (Directed Acyclic Graph) modes (for example, the Unexamined Japanese Patent Application Publication Nos. Hei5-260373 and Hei7-105020).
The Unexamined Japanese Patent Application Publication No. Hei5-260373 describes the digital picture signal processing apparatus capable of executing the high calculating process operation in high speeds by arranging this digital picture signal processing apparatus in such a manner that both the respective calculating process contents of a plurality of programmable calculating process units, and the connection modes of the respective programmable calculating process units by way of the network unit can be freely set via the host control means from the external unit. Thus, the digital picture signal processing apparatus may have high degrees of freedom with respect to changes in the functions and changes in the systems.
Further, the Unexamined Japanese Patent Application Publication No. Hei7-105020 describes the pipeline-formed image processing system capable of executing the image processing operation in the flexible manner, since the necessary functional modules are connected in the pipeline shape and initialized in the desirable sequence so as to execute the image processing operation. The operations of this pipeline-formed image processing system are carried out as follows.
In other words, when the connection of the necessary functional modules is completed, the acquisition of the header is requested with respect to the last-positioned functional module, if required. This request is sequentially traced through the connected modules, and is reached to the image input module located at the top, and the header information of the image to be read is returned. After each of the modules rewrites the information of the portion changed by the own module, this module transfers the rewritten information to the post-staged module. For instance, in such a case that a size of an input image is defined by (1000×1000) pixels, and a reducing process module capable of reducing the above-described image size into (500×500) pixels is contained in the processing pipeline, this reducing process module changes the size information (1000×1000 pixels) contained in the header information which is transferred from the pre-staged module into another size information (500×500 pixels), and, transfers this changed size information to the post-staged module.
While such an image processing operation is sequentially repeated, the last-positioned process module finally outputs to the external unit, the header information of the image which is outputted by this image processing pipeline. Next, the data processing operation is carried out based upon the acquired header information and the like. Similar to the header information processing operation, this data processing operation is performed in such a manner that when a constant amount of data is requested to be outputted with respect to the last-positioned module, this last-positioned module requires the pre-staged module to input such an image data required for this data processing operation. This request is traced back to the pre-staged module so as to be transferred to the image input module in a similar manner, the necessary image data is read out form the image input module, and, this read necessary image data is transferred to the post-staged process module. Then, while this image data is sequentially processed, the processed image data is finally outputted form the last-positioned process module.
When the above-explained image processing operation is carried out as to the final image, or the necessary image portion, since the processing pipeline is no longer required, the completion of the image processing operation is requested with respect to the last-positioned process module. Similar to the header information acquiring operation and the above-described process operation, this completion request is similarly traced back to the connections of these process modules, and thereafter, is reached to the frontmost image input module. Then, this image input module releases the resources employed in the process operation, and releases the own module, and returns the control to the post-staged module. This post-staged module to which the control is returned similarly performs such a process operation for releasing the resources and the own module, and further, returns this control to the post-staged module. At such a time instant when releasing of the resources/own module as to the last-positioned module is accomplished, all of the image processing operations are accomplished.
The above-described conventional technique described in the Unexamined Japanese Patent Application Publication No. Hei7-105020 implies that the simulation system for the multiple processing pipeline constructed by the single processor, which is utilized in such an operating system (OS) as UNIX (registered trademark), is applied to technical fields such as an image processing field. Then, more specifically, in this image processing field, since a single processing unit is limited to a portion of an image, for example, one line of an image is limited to be processed, each of the process modules can extremely reduce memory areas used to hold image-processed data.
As a result, this conventional technique can provide the low-cost image processing apparatus capable of executing the complex image processing operations with using the small memory capacity. Alternatively, in the case that such an image processing apparatus is operable on an operating system for supporting a virtual memory, since “swap out” caused by a shortage of memory capacity can be suppressed to a minimum value, this image processing apparatus can execute the image processing operation in a high speed.
Furthermore, as another conventional technique, the following image processing apparatus is proposed (see the Unexamined Japanese Patent Application Publication No. Hei8-272981). That is, for instance, in such a process operation that an entire input image is required in order to obtain 1 line of an output image, e.g., in a 90-degree rotating process operation, or in such a case of branching modules in which an input image is outputted to a plurality of process modules, when buffering of the entire image is ended, an ending process operation is carried out with respect to a pre-staged module. As a consequence, while a necessary memory resource can be suppressed to a minimum resource mount, the image can be processed in high speeds.
In the case that an image processing apparatus is arranged by which the desirable complex process operations can be carried out by combining the above-explained element/functional modules with each other in an arbitrary manner, as a very specific/important module, there is a branching module for outputting an input image to a plurality of processing modules. Although there are many process operations in which all of these process operations defined from an input process operation up to a final process operation may be carried out without executing a branching operation one time, such a branching module is required in the below-mentioned pipeline process operation.
For example, this pipeline process operation is realized in an adaptive filtering process operation in which filtering results obtained by executing a filtering process operation in different kernel sizes are compared with each other, and a filtering output is determined based upon this comparison result. Concretely speaking, as indicated in FIG. 12, while this process module contains an image input unit 111, an enlarging/reducing process unit 112, a color converting process unit 113, a branching process unit 114, filtering process units 115-1/115-2, and an output selecting unit 16, the adaptive filtering process operation is carried out with respect to image data which is stored in an image storage unit 100 and is to be processed.
In such a pipeline process operation, if the branching module is not used, as shown in FIG. 13, such a method is employed. That is, two sets of pipeline systems “A” and “B” are separately formed so as to perform the pipeline process operation, and a module for integrating these pipeline process results is formed. However, in the case of employing such an arrangement for separately forming the two pipeline systems, the completely same image input process, enlarging/reducing process, and color converting process operations are carried out two times respectively, as compared with such a case that the branching module of FIG. 12 is employed. As a result, large waste aspects will occur in view of resources/processing speeds.
Because of these reasons, the pipeline system constructed of the branching module, as indicated in FIG. 12, may constitute the effective method. The previously-explained Unexamined Japanese Patent Application Publication No. Hei7-105020 describes that branching of the pipeline system is performed, and/or the “fan-out” pipeline portion is present. However, detailed operations thereof are not described. Therefore, the following pipeline branching method is merely realized from the contents described in this publication. That is, while the entire image is buffered, the required intervening process results are merely outputted to the respective branching destinations.
Further, the above-described Unexamined Japanese Patent Application Publication No. Hei8-272981 discloses the following techniques. That is, by releasing the process module located at the pre-stage of the branching module, at least such a resource which has already become useless may be released. Also, while either the flag or the counter is employed which are used to check as to whether or not the processing operations are accomplished in all of a plurality of branched output destinations, if all of these process operations are accomplished, then the buffers and the like held by this branching module are released and the own branching module is released. However, as apparent from this operation explanation, the entire image corresponding to the intervening process results must be buffered in the branching module.
As previously described, although the branching module is very important in order that the arbitrary modules are combined with each other to realize the desirable complex process operation, the conventional techniques must buffer the entire image of the intervening process results obtained in the process modules up to the branching module, so that a large number of memories is necessarily required. As a result, this buffering necessity may cause one of factors for mitigating the merits of the pipeline type processing operation (including DAG type pipeline), namely highspeed processing operation can be carried out using a small number of memories.